1. Field of the Invention
The present invention relates to a system of detecting a road white line contained in a road image by processing the road image, a method for detecting a road white line contained in a road image by processing the road image and a storage medium storing a program for detecting a road white line contained in a road image by processing the road image.
2. Description of the Related Art
Research and development of technology for detecting a road white line are being pursued by which right and left road white lines indicating both ends of a travelling lane are to be detected from a road image obtained through an image pickup of a road in front of a vehicle by using a road image inputting device so that the technology can be applied to automatic travelling of an automobile. One problem which arises often when the road white line is to be detected is that a more distant portion, that is, a more upper portion on the road image is affected much by noise caused by insufficient resolution or images of other vehicles.
To solve this problem, methods are proposed in which the road image is divided into, at least, two regions positioned up and down and results obtained in a lower region where the road image is affected less by noise are used for detecting the road white line in an upper region on the road image. One example of these conventional methods is disclosed in Japanese Patent Publication No. Hei6-24035. In the method disclosed therein, a road image is divided into, at least, two regions including a lower and upper region and, to decide a pair of straight lines having a greatest probability of defining a travelling road, using known information about a width of the travelling road, by selecting out of many and unspecified groups of straight lines detected by Hough transformation after edge extraction in each of the two regions, a pair of straight lines to be defined in the upper region is selected from groups of straight lines passing near an upper end of the pair of straight lines defined already in the lower region.
In the road image, generally, since the road image in the upper region is more affected by noise and has less resolution than in the lower region, detection of the white line in the upper region is difficult. In the conventional technologies described above, by selecting the pair of straight lines to be defined in the upper region from groups of straight lines passing near the upper end of the pair of straight lines defined already in the lower region, the detection of the white line is made somewhat easy. However, since, in the upper region, the pair of straight lines having the greatest probability of defining the travelling road, using known information about the width of the road, are determined by selecting out of many and unspecified groups of straight lines detected by Hough transformation after the edge extraction in a same manner as in the lower region, an obtained image is not resistant against noise and the method is not effective for detection. If the road image is divided into three and more regions in particular, the road image in the most upper region is affected more by noise, it is very difficult to decide the pair of straight lines having the greatest probability of defining the travelling road by selecting out of many and unspecified groups of straight lines.
In view of the above, it is an object of the present invention to provide a system and method capable of reliably and effectively detecting a white line in an upper region of a road image by dividing the road image into, at least, two regions positioned up and down and using a result from detection of the white line in a lowest region where the road image is less affected by noise. It is another object of the present invention to provide a system and method capable of reliably and effectively detecting the white line in the upper region even if the road image is divided into many regions positioned up and down.
According to a first aspect of the present invention, there is provided a road white line detecting system for processing a road image to detect a white line in the road image including:
a first means for detecting left and right straight lines by which left and right white lines are approximated in a lowest region out of two regions positioned up and down obtained by dividing the road image;
a second means for calculating a horizontal line from a first vanishing point being an intersection of the left and right first straight lines; and
a third means for detecting straight lines by which the left and right white lines are approximated in a region other than the lowest region, by detecting an intersection based on a characteristic that the intersection of left and right straight lines, by which the left and right white lines are approximated, exists on the horizontal line in a region other than the lowest region in the road image.
In the foregoing, a preferable mode is one wherein the third means includes a means for detecting the second vanishing point on the horizontal line based on a characteristic that, at least, in a region being adjacent to the lowest region out of regions other than the lowest region in the road image, intercepts at lowest ends of left and right second straight lines by which left and right white lines in the region being adjacent to the lowest region are approximated, exist on the left and right first straight lines and the second vanishing point being an intersection of the left and right second straight lines exists on the horizontal line.
Also, a preferable mode is one wherein the third means includes a means for predicting an intersection of straight lines by which left and right white lines are approximated, in an upper region other than the lowest region and not being adjacent to the lowest region in the road image, based on an amount of displacement, on the horizontal line, between intersections of straight lines by which left and right white lines in two or more regions below the upper region are approximated.
Also, a preferable mode is one that wherein includes a means for extracting, from the road image, a sequence of points existing on left and right white lines as a sequence of white line featuring points, together with a white line filter value showing similarity to the white line featuring point and wherein the first means includes a means for detecting straight lines by which the sequence of white line featuring points is approximated using a least squares method, with weight assigned to the white line filter value, for left and right sides, as line approximation of white lines and wherein the third means includes a means for detecting the intersection, on the horizontal line, of left and right second straight lines by which the sequence of white line featuring points in the region being adjacent to the lowest region is approximated using the least squares method, for both left and right sides, with weight assigned to the white line filter value and using upper end points, in the lowest region, of the left and right first straight lines as intercepts at their lowest ends in the region being adjacent to the lowest region.
In the above configuration, if a road surface is planar, both an intersection obtained by extended straight lines by which left and right white lines in the lowest region out of regions obtained by dividing the road image including left and right white lines drawn in parallel on the road surface are approximated and the intersection obtained by extended straight lines by which left and right white lines in the upper region are approximated, exist on one line (in this specification, the one line is referred to as a xe2x80x9chorizontal linexe2x80x9d). Even when the road image is divided into more regions positioned up and down, intersections of extended straight lines by which left and right white lines in each region are approximated, exist on a same horizontal line and this is true even if the road surface curves. Though it is impossible that the road surface is completely planar in a real world, in a case of the road such as a highway having less slopes, no big error would occur even when the road surface seen within an image range obtained by an image pickup is approximated by a plane. According to the present invention, by detecting left and right first straight lines by which left and right white lines are approximated in the lowest region out of the two regions positioned up and down obtained by dividing the road image, by detecting the horizontal line being the intersection of left and right first straight lines and by detecting the intersection based on a characteristic that the intersection of left and right straight lines by which left and right white lines are approximated in the region other than the lowest region in the road image exists on the horizontal line, straight lines by which left and right white lines in regions other than the lowest region are approximated, are detected. This allows the white line in the region other than the lowest region to be detected only by calculating position of the horizontal line already detected, thus enabling reliable and effective detection of the white line in the region other than the lowest region in the road image.
According to a second aspect of the present invention, there is provided a road white line detecting method for processing a road image to detect a white line in the road image including:
a step (a) of detecting left and right first straight lines by which left and right white lines are approximated in a lowest region out of, at least, two regions positioned up and down obtained by dividing the road image;
a step (b) of calculating a horizontal line from a first vanishing point being an intersection of the left and right first straight lines; and
a step (c) of detecting straight lines by which left and right white lines are approximated in regions other than the lowest region, by detecting an intersection based on a characteristic that the intersection of left and right straight lines by which left and right white lines are approximated in the region other than the lowest region in the road image, exists on the horizontal line.
In the foregoing, a preferable mode is one wherein the step (c) includes a step of detecting a second vanishing point on the horizontal line based on a characteristic that, at least, in a region being adjacent to the lowest region out of regions other than the lowest region in the road image, intercepts at lowest ends of left and right second straight lines by which left and right white lines in the region being adjacent to the lowest region are approximated, exist on the left and right first straight lines and that the second vanishing point being the intersection of the left and right second straight lines exists on the horizontal line.
Also, a preferable mode is one wherein the step (c) includes a step of predicting an intersection of straight lines by which left and right white lines are approximated, in an upper region other than the lowest region in the road image and not being adjacent to the lowest region, based on an amount of displacement, on the horizontal line, between intersections of straight lines by which left and right white lines in two or more regions below the upper region are approximated. A reason that the above method is possible is as follows. That is, if the image region is divided into more regions, in higher regions, the road image is affected more by noise. In a road such as a highway, a ratio of curvature of a curve is relatively large and the ratio of the curvature does not change at some midpoint of the curve. Therefore, in such the road, there appears only one curve in the road image and its ratio of the curvature is approximately constant.
Also, a preferable mode is one that wherein includes a step of extracting a sequence of points existing on left and right white line image, from the road image, together with a white line filter value showing similarity to the white line featuring points as a sequence of white line featuring points, and wherein the step (a) includes a step of detecting straight lines by which the sequence of white line featuring points is approximated using the least squares method, with weight assigned to the white line filter value, as line approximation for left and right sides and wherein the step (c) includes a step of detecting an intersection, on the horizontal line, of left and right second straight lines by which the sequence of white line featuring points in the region being adjacent to the lowest region is approximated by using the least squares method, for both left and right sides, with the white line filter value assigned and using upper end points of the left and right first straight lines in the lowest region as intercepts at their lowest ends in the region being adjacent to the lowest region.
According to a third aspect of the present invention, there is provided a storage medium storing a program for detecting a road white line, which is a machine-readable storage medium, to process a road image, to detect a white line in the road image and to cause a computer to carry out the following steps:
(a) detecting left and right first straight lines by which left and right white lines are approximated, in a lowest region out of, at least, two regions positioned up and down obtained by dividing the road image;
(b) calculating a horizontal line from a first vanishing point being an intersection of the left and right first straight lines; and
(c) detecting straight lines by which left and right white lines are approximated, in regions other than the lowest region, by detecting an intersection based on a characteristic that the intersection of left and right straight lines by which left and right white lines are approximated in the region other than the lowest region in the road image, exists on the horizontal line.
In the foregoing, it is preferable that the step (c) includes a step of detecting a second vanishing point on the horizontal line based on a characteristic that, at least, in a region being adjacent to the lowest region out of regions other than the lowest region in the road image, intercepts at lowest ends of left and right second straight lines by which left and right white lines in the region being adjacent to the lowest region are approximated, exist on the left and right first straight lines and that the second vanishing point being an intersection of the left and right second straight lines exists on the horizontal line.
Also, it is preferable that the step (c) includes a step of predicting an intersection of straight lines by which left and right white lines are approximated, in an upper region other than the lowest region in the road image and not being adjacent to the lowest region, based on an amount of displacement, on the horizontal line, between intersections of straight lines by which left and right white lines in two or more regions below the upper regions are approximated.
Furthermore, it is also preferable that the program includes a program to cause the computer to carry out a step of extracting, from the road image, a sequence of points existing on left and right white lines as a sequence of white line featuring points, together with a white line filter value showing similarity to the white line featuring points and wherein the step (a) includes a process of detecting straight lines by which the sequence of white line featuring points is approximated using the least squares method, with weight assigned to the white line filter value, for left and right sides, as line approximation of white lines and wherein the step (c) includes a process of detecting an intersection, on the horizontal line, of left and right second straight lines by which the sequence of white line featuring points in the region being adjacent to the lowest region is approximated using the least squares method, for both left and right sides, with weight assigned to the white line filter value and using upper end points, in the lowest region, of the left and right first straight lines as intercepts at lowest ends in the region being adjacent to the lowest region.